Skip to content

docs: clarify debouncing timing and preprocessor interaction#1353

Merged
rubenfiszel merged 1 commit intomainfrom
docs/debounce-preprocessor-clarification
Apr 15, 2026
Merged

docs: clarify debouncing timing and preprocessor interaction#1353
rubenfiszel merged 1 commit intomainfrom
docs/debounce-preprocessor-clarification

Conversation

@rubenfiszel
Copy link
Copy Markdown
Contributor

Summary

Documents that for flows with a preprocessor, debouncing is evaluated after the preprocessor runs (against its output), and calls out the consequences users most commonly miss.

Motivation

Users hitting non-debouncing symptoms (e.g. on Kafka triggers with `auto_commit=false` + preprocessor + debouncing) usually don't realize that:

  1. The preprocessor runs on every incoming call — it is never debounced.
  2. The accumulation field (`debounce_args_to_accumulate`) must exist in the preprocessor's output, not the raw trigger event.
  3. Under the current default key semantics, every non-accumulated field in the preprocessor output is part of the debounce key. So if the preprocessor emits a per-call-varying field — a timestamp, a kafka offset, a request ID, or the raw event via `{...event, items}` — each call gets a unique key and debouncing silently does nothing. The fix is either to keep the preprocessor output stable across calls or to set an explicit custom debounce key template.

Changes

  • `docs/core_concepts/22_job_debouncing/index.mdx`:
    • New When does debouncing run? section distinguishing the push-time path (scripts / flows without preprocessor) from the post-preprocessing path (flows with preprocessor), with bullets covering the three pitfalls above.
    • Added a paragraph under Debounce args to accumulate reiterating that, for flows with a preprocessor, the accumulation field must exist in the preprocessor's output and that other emitted fields need to be stable (or a custom key must be set).

Test plan

  • Render the docs site locally and verify the new section appears in `/docs/core_concepts/job_debouncing`
  • Verify both internal links resolve: `../43_preprocessors/index.mdx` and the `#custom-debounce-key` anchor

Generated with Claude Code

Explain that for flows with a preprocessor, debouncing is evaluated
after the preprocessor runs (not at push time) against the preprocessor's
output. Call out the consequences:

- The preprocessor itself is never debounced; every incoming call runs it.
- The accumulation field must exist in the preprocessor's output.
- Non-accumulated fields in the preprocessor output participate in the
  default debounce key, so per-call-varying fields like timestamps,
  kafka offsets, or request IDs silently break debouncing — fix by
  keeping the preprocessor output stable or setting an explicit custom
  debounce key template.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@rubenfiszel rubenfiszel marked this pull request as ready for review April 15, 2026 21:20
@rubenfiszel rubenfiszel merged commit f89e18b into main Apr 15, 2026
8 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant